Networks often have latency and bandwidth limitations that can be overcome using a number of methods. Some methods include using transaction accelerators. For example, McCanne I, McCanne III and McCanne IV describe how a pair of transaction accelerators can improve performance over a section of a network. In a general example, a client communicates with a server over a network wherein the path of traffic between them, at least in part, travels between transaction accelerators. Thus, a client would be coupled to a client-side transaction accelerator, which would be coupled to a portion of the network, which would be coupled to a server-side transaction accelerator that is in turn coupled to the server. In some instances, the portion of the network between the accelerators is a wide area network (WAN).
Transaction accelerators can cooperate to accelerate client-server transactions across a network. Unless otherwise indicated, it should be understood that the transaction accelerators could operate symmetrically and not need to take into account which of a pair of transaction accelerators is closer to the client and which is closer to the server. The roles of client and server are typically determined by which entity initiated a network connection, with the initiator being labeled the client and the other end being labeled the server. It should also be understood that a transaction accelerator pair might support more than one client and more than one server. While a network path is often described herein as having a transaction accelerator pair in the path, unless otherwise indicated, such description should not be construed as limiting to having only two transaction accelerators in any given path. Also, there are some instances where traffic might pass through one of the transaction accelerators and end up at the other end without having passed through the other transaction accelerator of the pair, however where the first transaction accelerator performs a transformation that is expected to be inverted by the other transaction accelerator, it should pass to the other transaction accelerator if data is to be received at the ultimate destination in a transparent manner.
In a transaction accelerator process, there can be multiple connections, e.g., multiple network connections used to transport data between the client and the server, such as a first network connection between the client and the client-side transaction accelerator, a second network connection between the server and the server-side transaction accelerator and a third connection between the client-side transaction accelerator and the server-side transaction accelerator. The first and second network connections are collectively referred to herein as an “outer connection” and “inner connection” refers to the third network connection. Where the transaction acceleration is transparent to the client and server, the outer connection carries unaccelerated traffic, while the inner connection carries accelerated traffic. This works well where, for example, the transaction accelerators are positioned at WAN-LAN boundaries such that the first and second connections are LAN connections and the third connection (the inner connection) is a WAN connection.
The outer connection can be arranged so that it appears to be a single logical connection from client to server, with its behavior identical to, or substantially similar to, the behavior of a client/server connection without either transaction accelerator present. In that case, the existence of the inner connection is not “visible” to the client or server. As a result, the transaction accelerators can be free to use varied acceleration and optimization methods and techniques for traffic on the inner connection, including dynamically changing techniques, without affecting any aspect of how the client or server is configured.
The accelerated traffic on the inner connection is usually substantially different from the unaccelerated traffic on the outer connection. In particular, the accelerated traffic is often transformed to contain payload data or whole messages that appear entirely different from the original unaccelerated traffic. Since the transformed traffic only needs to flow between transaction accelerators and not directly to a client or server, the inner connection is often most simply and reliably implemented as a simple connection between transaction accelerators and with that, different clients, servers, and/or traffic types can all be treated alike.
However, where the traffic is undifferentiated on the inner connection, network tools (such as network monitoring tools, network processing tools, network debugging tools, etc.) might have trouble if they expect differentiated packets/data/traffic differentiated by client, server, ports, traffic type, etc. For example, some network tools distinguish traffic using network attributes of the client and/or server, such as network address and/or port. Examples of monitoring and processing that such tools can perform include measurement of traffic levels, classification of traffic, enforcement of policies such as Quality of Service (QoS) on traffic, filtering, and access control lists.
Such monitoring or processing can be done normally if it takes place on the outer connection because traffic on the outer connection can be expected to have the same network attributes that client/server traffic would have without transaction accelerators present. However, network tools might not work as well if they are applied to the inner connection, where the traffic looks much different and might not be as differentiated as needed by the network tools. In some instances, it would be desirable to use the network tools on the inner connection as well as the outer connection.
There are some existing approaches to using network tools on transformed traffic. For example, network tools that analyze Netflow information (Netflow was developed by Cisco Systems of San Jose, Calif., USA) can deal with Netflow information about the inner connection provided by transaction accelerators, but that is limited to Netflow-based monitoring.
“Port mapping” can be used to differentiate traffic flows, by partitioning traffic between transaction accelerators into a number of equivalence classes of connections, each of which uses a particular port number. Some port mapping techniques have been developed by Riverbed Technology, of San Francisco, Calif., USA, the current assignee of the present application. Network tools that perform monitoring or processing based on port number can take advantage of port mapping. Port mapping can be used to distinguish among traffic based on non-port attributes, but only by mapping those attributes onto ports. For example, traffic to destination A can be distinguished from traffic to destination B, but only by setting up rules so that traffic to A is sent on port PA while traffic to B is sent on port PB. In addition, the monitoring or processing mechanisms must operate in terms of the ports used, rather than the actual source/destination address information.
Yet another example of an approach is to use a “router transparency mode”, such as is provided by products developed by Expand Networks of Roseland, N.J., USA. With a router transparency mode, the original (outer-connection) IP and TCP or UDP headers are reused for packets on the inner connection. While this approach is much more general, and is usable for a wide variety of processing and monitoring mechanisms on the inner connection, the approach only works correctly if 1) the routing system delivers a packet to the counterpart transaction accelerator even though the packet is addressed to another entity (the client or server beyond the counterpart transaction accelerator); and 2) the counterpart transaction accelerator correctly handles the reverse transformation of the packet even though the packet is addressed to another entity. If these conditions are not met, the mode does not work very well.
If routing changes mean that the packet is actually delivered to its stated destination, the resulting problems are hard to troubleshoot, since the packet did not come from its stated source, it was not intended for its stated destination, and it likely contains data that does not match the formats usually communicated on its stated port.
In view of the existing solutions, what was found to be needed are methods and apparatus for a general facility for processing or monitoring traffic and operating other network tools, but providing better characteristics, such as allowing for better troubleshooting characteristics.